This is achieved via (1) fitting a parameterized material engineering domains where online model on such past data relating the material input experimentation to collect data is too expensive with its output properties; and (2) finding an input optimizer and needs to be replaced by an in silico maximization with respect to the learned parameterization. of a surrogate of the black-box function. Although such a surrogate can be learned from Naively, such in silico approach would trivialize the optimal offline data, its prediction might not be reliable design problem into a vanilla application of gradient outside the offline data regime, which happens ascent and supervised learning. However, in practice, the when the surrogate has narrow prediction margin prediction of such vanilla surrogate might not be reliable and is (therefore) sensitive to small perturbations outside the offline data regime (Fannjiang & Listgarten, of its parameterization. This raises the following 2020). Often, its prediction can become highly erratic at questions: (1) how to regulate the sensitivity out-of-distribution data regimes, misguiding the optimization of a surrogate model; and (2) whether conditioning process toward sub-optimal candidates. This happens an offline optimizer with such less sensitive when the surrogate has narrow prediction margin at those surrogate will lead to better optimization performance.

Learning graphical conditional independence structures is an important machine learning problem and a cornerstone of causal discovery. However, the accuracy and execution time of learning algorithms generally struggle to scale to problems with hundreds of highly connected variables -- for instance, recovering brain networks from fMRI data. We introduce the best order score search (BOSS) and grow-shrink trees (GSTs) for learning directed acyclic graphs (DAGs) in this paradigm. BOSS greedily searches over permutations of variables, using GSTs to construct and score DAGs from permutations. GSTs efficiently cache scores to eliminate redundant calculations. BOSS achieves state-of-the-art performance in accuracy and execution time, comparing favorably to a variety of combinatorial and gradient-based learning algorithms under a broad range of conditions. To demonstrate its practicality, we apply BOSS to two sets of resting-state fMRI data: simulated data with pseudo-empirical noise distributions derived from randomized empirical fMRI cortical signals and clinical data from 3T fMRI scans processed into cortical parcels. BOSS is available for use within the TETRAD project which includes Python and R wrappers.